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Teachers are expected to engage in the praxis of educational reform, and one 
of the resurgent interests in the field of education is the conduct of action 
researches. In the Philippines and in many parts of the world, teachers are 
encouraged to perpetuate a culture of conducting action researches to 
improve the teaching and learning process. Considering the scant literature 
towards evaluation criteria for quality action researches, this exploratory- 
sequential mixed method study aims to determine the qualities that research 
experts perceive to be useful in evaluating action researches. Using Analytic 
Hierarchy Process, which is a multiple decision-making algorithm, research 
experts asserted that among the identified evaluation criteria, quality action 
researches should underscore its contribution to theory and practice, and 
require actionability and reflexivity among researchers and participants. The 
findings of this study may be of considerable interest and use to teachers, 


research authorities, and organizations who would like to ensure that teachers 
produce quality action research to improve educational practices. 
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1. INTRODUCTION 

There is a resurgent interest in action research particularly in the field of education nowadays. 
Conducting quality researches does not only inform teaching and learning but empowers teachers to become 
more reflective of their roles and responsibilities in the clasroom, and for schools to improve the delivery of 
quality education for the students. It is for this reason that teachers, if not required, are encouraged to conduct 
researches which are relevant to their field of teaching in the form of action research. 

Action research is a cyclical process that may be used to improve instructional practice, assessment 
tools, and student outcomes with focus on specific problems which usually take place in the natural setting of 
a classroom, laboratory or school [1]. This is the main reason why Lytle & Smith [2] use the term teacher 
research synonymously with action research where teachers are situated as learners in order to change and 
improve their teaching [3]. It is not only a scientific way of producing knowledge but a systematic way of 
solving problems [4]. The advantage of this kind of research compared to the other kinds of traditional 
researches, aside from its simplicity, is its potential to create meaningful and authentic change for those 
involved, whether in a classroom or community [5]. In the Philippines, teachers of private and public sectors 
are enjoined to conduct action researches which can improve their teaching practices and learning outcomes 
of students. 
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The term “quality” as operationally used in this study is defined as the goodness of an action 
research based on some standards. Ensuring that teachers conduct quality action researches is of great 
importance. While action research is not a recent approach to research and that it has been widely used in 
various disciplines, there is an immense literature gap regarding on how to evaluate action researches in 
education, and what criteria are relevant to use. According to Meyer [6], the fact that action research arises 
from a different epistemological background means that it cannot be judged using the same criteria as other 
research approaches. Moreover, the fact that there is no convention for evaluating action researches [7] 
particularly in education necessitates collective effort to identify criteria which can be used evaluate action 
researches made by teachers. In fact, one of the implications in the study of Rowell, et. al [8] reveals that one 
of the challenges which is associated with action research is contribution to quality and rigor in critiquing 
action research. Teachers need further support and clarification in action research [9] most notably because 
evaluation of researches is usually set by the examining authority in different academic institutions in 
the Philippines, and this process from identification of criteria to the actual evaluation of action researches 
may be subjected to biased and poor judgment, if not done correctly and conscientiously. 

One of the notable literatures related to the present study is what Zuber- Skerrit & Fletcher [10] 
presented as quality criteria in action research theses in social sciences. According to the researchers, 
a quality action research presents a critical analysis of a well-defined problem which raises a “thematic 
concern” referred to as a major issue in community development. Also, the “quality” comes with the use of 
an appropriate methodology which can advance knowledge in the field. It provides convincing evidence 
for the claimed knowledge and demonstrates critical reflection which can contribute to transforming 
knowledge and practices. In addition, valid and quality researches emerge from the reflection of teachers 
on their work [11]. 

Moreover, Feldman [12] criticized the principles of quality action research proposed by Heikkinen, 
Huttunen and Syrjala. According to him, historical continuity, reflexivity and dialectics, workability and 
evocativeness may be insufficient criteria because these may only applicable to the research report and not to 
the action research itself. While he thought that reflexivity and dialectics may be used as criteria, he added 
that action researches should include clear and detailed descriptions of how and why the data were collected, 
provide clear and detailed descriptions of how their narratives were constructed from the data, present 
an explanation or theory of why it works. Moreover, he averred that it is when the action researchers pay 
attention to validity of their work that their action researches can have good quality. 

The purpose of this paper is to determine the preferred evaluation criteria for action researches in 
education using the Analytic Hierarchy Process (AHP) by experts. This method is a multiple decision- 
making method that models decision-making processes mathematically and are used to solve complex 
problems [13]. Using AHP, this research intends to identify, rank, and determine the weight of the criteria 
which research experts perceived as critical in appraising action researches conducted by teachers. This study 
is of great relevance since evaluation of action researches made by teachers does not only improve 
the delivery of education, but it used nowadays to appraise teachers’? dedication and commitment for 
professional growth. 


2. RESEARCH METHOD 

This study employed an exploratory- sequential mixed method design. The purpose of 
an exploratory- sequential mixed methods design involves the procedure of first gathering qualitative 
data to explore a phenomenon, and then collecting quantitative data to explain relationships found in 
the qualitative data [14]. The researcher analyzed documents particularly rubrics used in assessing action 
researches by experts and organizations, reviewed literature about action researches, and interviewed field 
experts to come up with evaluation criteria. These were then tested and further analyzed using AHP to 
determine the criteria weights as evaluated by the research experts. 

The main respondents of this study were five experts in the field of research and evaluation of who 
were purposively selected by the researcher. In selecting respondents, four criteria were considered which are 
their academic preparation, years in the teaching profession, and experiences in writing and evaluating 
researches. All five experts are Ph.D. in Educational Research and Evaluation candidates, have at least 10 
years of teaching experience, have published researches both in local and international journals, and were 
invited to evaluate researches and journals a number of times in the past. Figure 1 describes process flow of 
this research. 
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Figure 1. Process flow of this research 


2.1. Collection of qualitative data 

The first phase of data gathering was the collection and analysis of qualitative data. The researcher 
conducted content analysis of evaluation rubrics used by ten (10) organizations in assessing action researches 
made by teachers, interviewed research experts, and reviewed relevant literature. The data obtained from 
multiple sources of data were validated by two experts who both have PhD. in Education degrees. 


2.2. Collection of quantitative data using analytic hierarchy process 

After finalizing the criteria for the evaluation of educational action researches, it was subjected to 
AHP. This is a multi-criteria programming technique for decision making in complex environments in which 
many variables or criteria are considered in the prioritization and selection of alternatives or projects such as 
evaluation criteria for educational action researchers where there are a number of considerations which need 
to take into account. AHP transforms the comparisons, which are most often empirical, into numerical values 
which are further processed and compared. The weight of each factor allows the assessment of each one of 
the elements inside the defined hierarchy. This capability of converting empirical data into mathematical 
models is the main distinctive contribution of the AHP technique when contrasted with other comparing 
techniques [15]. 

The main advantage of AHP is its ability to rank choices in the order of their effectiveness in 
meeting conflicting objectives [16] which is suitable to the purpose of this study which is to determine 
evaluation criteria for educational action researches made by teachers. Moreover, AHP is appropriate to 
people are working with problems involving human perceptions and judgment whose resolutions have long- 
term repercussions [16] and allows some small inconsistencies in the judgment because it recognizes that 
human perception is not always consistent. In addition to this, the implementation of AHP is based on the 
experience and knowledge of the experts or users to determine the factors affecting the decision process [17]. 
In this study, after the criteria in evaluating action researches were identified based on the qualitative analysis 
of data, the magnitude or weight of each criterion were determined by comparing two criteria at a time. AHP 
requires decision makers to carry out simple pairwaise comparison judgments [18]. This stems from the fact 
that AHP adheres the fact that the magnitude of criteria may not be equal as preferred by various evaluators 
or experts. In pairwise comparison, the respondents were asked to use the AHP scale that follows in 
determining which of the criteria has more merit for them as shown in Table 1. 


Table 1. The analytic hierarchy process (AHP) scale 








Intensity of Importance Short Descriptors Interpretation 
1 Equal The two criteria contribute equally to the objective 
3 Moderate Experiences and judgments slightly favor one over the other. 
5 Strong Experiences and judgments strongly favor one over the other 
7 Very Strong Experiences and judgments very strongly favor one over 
the other where its importance is demonstrated in practice. 
The evidence favoring one over the other is of the highest 
9 Extreme : as 
possible validity. 
2, 4, 6, 8 Intermediate Values When compromise is needed 





3. RESULTS AND DISCUSSION 

After conducting content analysis of the evaluation rubrics from ten research experts and 
organizations, and themes surfaced from interviews, seven research criteria emerged. The most 
comprehensive summary of evaluation criteria for action researches which can encapsulate the results of the 
qualitative analysis done by the researcher can be compared to what Bradbury-Huang [19] presented as 
quality criteria for action researches. For the purpose of clarity and proof of literature underpinnings, these 
seven criteria were adopted in this study. These criteria were compared and analyzed further by the research 
experts to determine its weight using the Analytic Hierarchy Process. 
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3.1. Well-articulated objectives 

Quality action research has clear objectives which guided the entire work from the beginning until 
the end. This means that a well-defined and strong purpose is articulated from the beginning of the study 
which gives direction to the action research process. At the end of the study, the action plan and the impact of 
the involvement of teachers in the process can be asserted as the realization of the stated objectives. 


3.2. Incorporate partnership and cooperation among stakeholders 

Quality action research encourages participation among stakeholders from mere consultation with 
them and to the extent of engaging them as co-researchers. The identified and raised problems in action 
researches may directly affect the stakeholders such as the students, teachers, administrators, parents, and 
the whole school community. It is essential to encourage them to be actively engage in every step of 
the research process as they help the researcher in the identification of the problem, design of 
the investigation, and in the implementation and assessment of the action plan. In fact, Hart & Bond as cited 
in Deery [20] asserted that the nature and degree of democratic collaboration are crucial to the success of 
the action research process. 


3.3. Contribution to theory and practice 

Quality action research does not only contribute to the action research literature but provides inputs 
to theories and practices in education. The contribution to theory and practice is simply the impetus which 
will help change and improve the present problematic situation in the classroom or school setting. The aim is 
for improvement within the context and situation within which the study is conducted in the key priority of 
the action. With this, Frogatt & Hockley [21] seconded that this kind of methodology bridges the gap 
between theory and practice where teachers are challenged to leverage their insider knowledge to change and 
improve classroom practices [22]. 


3.4. Use of appropriate and accurate methods and processes 

Quality action research makes use of appropriate and accurate methods and processes which help 
the researchers in realizing the objectives of the study. Just like traditional scientific researches, the paradigm 
of action researches needs to make use of accurate methodologies. According to Bradburry and Reason as 
cited in Skerrit and Fletcher [10], drawing on and integrating diverse ways of knowing and using different 
methodologies appropriately and creatively in the context of action research is an indispensable expectation 
to this kind of research. 


3.5. Requires actionability 

Findings generated from action research provide specific actions in response to the problem raised 
in the study. As posited by Cain [2], research and action are integrated in a series of flexible cycles. 
The planning, introduction, and implementation of the ‘action” are to address the raised problem is needed to 
bring about positive change. 


3.6. Requires reflexivity among action researchers 

The action research becomes an impetus to make the researcher takes a personal, involved, and self- 
critical stance about their role in from the conceptualization of the problem and in designing action to 
promote change and improvement. Giddens as cited in Robertson [23] described reflexibity as self- 
awareness. These quality criteria can be demonstrated by action researchers by presenting how their 
involvement in the research process has made them more aware of their contribution to the problem and 
transformed them to improve their roles as educators to address the problem. Moreover, as researchers, 
they should ardently show how their engagement in the process of action research has penetrated in their 
personal life and work, and how their social world affects their disposition and action leading to 
the ariculation of the problems and formulation of solutions. 


3.7. Significance beyond immediate context 

Action researches of good quality are not only relevant to the immediate context but extend to 
bigger systems and communities where findings can be of great use. While the priority of action researches is 
to test and recommend possible solutions to problems of immediate context such as those affecting a specific 
classroom setting, its significance should go beyond this classroom and should benefit other classrooms of 
almost the same setting or bigger entity where this classroom is experiencing academic challenges. This is 
what Huang- Bradburry [19] described as the meaning and support of action research to the flourishing of 
people, communities, and wider ecology. 
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Table 2 summarizes the derived weights of each evaluation criterion which is the result of 
the Analytic Hierarchy Process pairwise comparison. Among the seven evaluation criteria, the criterion 
contribution to theory and practice obtained the highest mean weight of 24.64% with three experts perceiving 
it as the most important aspect to look into action researches conducted by teachers. It is followed by 
the criterion where action research should require actionability with a mean weight of 18.50% with one 
expert believing that it is the most important criterion. The third highest criterion is that action research 
requires reflexivity among action researchers with a mean weight of 16.60 with one expert believing it is 
the most important evaluation criterion for action research. 


Table 2. AHP derived weights of evaluation criteria for action research 
Experts 








Criteria 1 2 3 4 5 Mean Rank 

1. Well -articulated Objectives 2.60 3.8 2.5 5.5, 10.7 5.04 7 
2. Incorporate partnership and cooperation 13.50 35 36 35 18.8 8.56 6 
among stakeholders 
3. Contribution to theory and practice 8.20 36.9 19.4 28.3 30.4 24.64 1 
4. Use of appropriate and accurate 8.00 13.1 15.8 19.0 70 12.58 5 
methods and processes 
5. Requires actionability 43.20 6.1 11.4 13.0 18.8 18.50 2 
6. Requires reflexivity among action 2.10 25.7 323 13.2 9.8 16.60 3 
researchers 
7. Significance beyond immediate context 22.40 10.9 15 17.5 4.5 14.08 4 

AHP Consistency Ratio 0.098 .098 .084 .086 .093 





While all criteria are proven to have wide applicability in evaluating action researches in education 
based on the first phase of data collection which is content analysis of evaluation rubrics used by different 
organizations, the criteria significance beyond immediate context (14.06%), use of appropriate and accurate 
methods, and processes (12.58%), incorporate partnership and cooperation among stakeholders (8.56%) 
and well-articulated objectives (5.04%) were given less priorities by the research experts. Since all 
the consistency ratios are less than 10%, the inconsistencies of the subjective judgments of the experts 
are acceptable. 

In order to enrich and elucidate the data obtained from AHP pairwise comparisons made by experts, 
interview data were considered. According to the first expert (E1), “Since action research has many different 
frameworks and paradigms, some evaluation criteria may not be explicitly evident in the report such as 
partnership and cooperation among stakeholders because of issues of time and resources or may not be able 
to address its significance beyond immediate context because of lack of generalizability power. Instead, what 
must be ensured is its contribution to theory and practice in education which is to improve educational 
practice. This is supported by E5 when he posited that the criterion contribution to theory and practice is 
the ultimate goal of conducting researchers of whatever forms. It is likened to an umbrella criterion that 
covers the other identified criteria. These ideas support that the very essence of action research in education 
is to provide practitioners with new knowledge and understanding on how to improve practices by resolving 
significant problems in classrooms and schools [24]. In action research, teachers prescribe a recommendation 
written and prepared by them which they believed are practical solutions to their problems [25]. 

El who predominantly gave the highest priority weight on the criterion requires actionability 
asserted that action should be an intrinsic part of the researcher(s) and participants which is seen through 
the action taken to look at one’ s own practice or situation to seek out aspects in teaching to improve practice 
or situation. According to the expert, when action research requires actionability, it does not only describe 
and require the final action prescribed to solve the raised problem, but it highlights the series of critical and 
careful actions taken by the researchers and those who are involved in every step of the research process to 
realize the objectives of the study. This disposition supports that quality action researches not only inform 
participants and audience, but encourages them to carry out actions to improve their present condition as 
change agents [19] and these actions are substantive located through a research procedure, disciplined by 
enquiry, and a personal attempt to understand, improve, and reform practice [26]. This makes the “action” in 
action research challenging to define, measure, and achieve [27]. 

Requiring reflexivity among action researchers and participants is also given higher priorities by 
the experts among others. According to E3, quality action researches in education should be informed by 
reflexive thinking and investigation, a form of self-study of one’s own roles and stance of the problem and 
how his present performance can be improved to change the situation for the better. The expert stressed that 
as teacher-researcher, one needs to unpack the very self in the practice of teaching, engage in critical and 
careful thinking to self-evaluate current practice. Hong & Lawrence [28] renounced the same when they 
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contend that self-reflection is important in action research as it is a powerful way to know more about the self 
in research and practice a reflexive pedagogy which can help teachers closely examine current practices and 
spearhead change as teacher- leaders. By being reflexive in their approach, teachers are empowered when 
they are able to collect and use data in making informed decisions about their own schools and classrooms, 
Book; Fueyo & Koorland; Hensen; as cited in Hine [24]. This is true when teachers exercise their individual 
talents, experiences and creative ideas within the classroom; they become empowered to make changes 
related to teaching and learning. Also, the self-reflexivity is seen in a spiral approach which consists of 
a planning, implementing action, reflecting and repeating the process as necessary [29]. This makes 
reflection a complex process which demands the role of values and existential knowledge in education 
among teachers [30] and researchers should continuously reflect and do self-evaluation which should be 
intentional and systematic [31]. 


4. CONCLUSION 

The proclivity of teachers to conduct researches for personal and professional growth is very evident 
at present as action researches continue to score and flourish. Hence, the present research focused on 
articulating the qualities of action researches perceived to be meaningful and critical as perceived by experts. 
With the complexity and variations in the framework, theoretical underpinnings, and contextual 
considerations, the researcher recognized that quality in action research is multifaceted, and that teachers, 
evaluators, and organizations need to critically select what qualities should be prioritized. 

The qualitative empirical data collected in the first phase of this exploratory- sequential mixed 
method design study has proven that there are many possible quality criteria which can be used in evaluating 
action researchers made by teachers such as what Brandburry-Huang summarized in her paper. The use of 
the Analytic Hierarchy Process in the second phase of data gathering and analysis facilitated, clarified, and 
refined which of these criteria may be given more consideration and greater weights such as the criteria of 
(1) contribution to theory and practice, (2) requires actionability, and (3) reflexivity among researchers. 

The contribution of action research to educational theory and practice should always be 
foregrounded regardless of research objectives, context, and methods. This means that the action researcher 
is expected to impress the quality of his/her work by underscoring clearly how the entire process can be 
meaningful by expanding existing knowledge and improve current practice. Moreover, the action in action 
research brings quality when it is based on the systematic process of research and is taken seriously and 
effectively to improve the teaching and learning condition. Also, quality of reflexivity among action 
researchers cannot be taken for granted. Teachers should engage in reflective teaching habitually, and find 
ways on how their roles can be modified and improved to espouse change and solution to pressing problems 
inside the classroom and within the school. 

While qualities in action research continue to evolve and change in the future, the Analytic 
Hierarchy Process has proven that it can be used to help teachers, practitioners, and evaluators to select and 
make complex decisions in dividing priorities among seemingly equal valued options concerning research 
and evaluation. The findings of this study may be of considerable interest to those individuals and 
organizations wishing to evaluate the merit and quality of action researches completed by teachers. 
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